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ABSTRACT 


Citizens  are  understood  as  the  ultimate  client  of  government,  development  institutions  and  private  sector  interventions  in  a country.  Citizens  can  act  as  individuals  or 
organize  themselves  in  associations  and  groups  such  as  community  based  groups,  women's  groups,  or  indigenous  peoples'  groups.  Civil  society  organizations  (CSOs) 
can  represent  citizens  and  can  include  organizations  outside  the  public  or  for-profit  sector,  such  as  nongovernmental  organizations  (NGOs),  charitable  organizations, 
faith-based  organizations,  foundations,  academia,  associations,  policy  development  and  research  institutes,  trade  unions,  and  social  movements.  In  this  context,  the 
term  citizen  is  not  used  in  a legal  sense  but  is  understood  in  the  broad  sense  of  referring  to  all  people  in  a society  or  country  in  an  inclusive  and  nondiscriminatory  way. 
We  leverage  the  deep  penetration  of  mobile  phones  in  Telangana  to  design  a suite  of  IVR  (Interactive  Voice  Response)  tools  that  can  help  capture  community  percep- 
tions, improve  awareness  of  the  people,  and  verify  official  records  directly  by  the  beneficiaries  themselves.  In  the  context  of  a rural  employment  guarantee  scheme,  we 
evaluate  these  tools  in  a few  villages  in  the  state  of  Telangana  and  demonstrate  that  there  is  good  scope  for  using  IVR  tools  to  serve  as  a citizen  engagement  channel  for 
welfare  schemes.  Our  contribution  lies  in  outlining  several  use-cases  for  technology  interventions,  and  uncovering  nuances  that  should  be  addressed  if  such  systems 
are  integrated  into  the  implementation. 

KEYWORDS:  Information  technology,  use-cases,  IVR  Data  base  Technology. 


1.  INTRODUCTION 

The  state  governments  in  Telangana  run  various  welfare  schemes  like  the  civil 
society  organizations  (CSOs)  and  the  non-  governmental  organizations  (NGOs). 
However,  the  governments  do  not  have  any  direct  mechanism  to  understand  pub- 
lic perception  on  these  schemes,  the  level  of  awareness  about  these  schemes  and 
problems  that  beneficiaries  face  in  the  implementation.  Studies  show  very  low 
awareness  on  (CSOs)  among  the  beneficiaries,  and  other  studies  show  misuse  of 
funds.  Systems  like  call  center  hiplines  set  up  by  the  governments  are  far 
underutilized  largely  owing  to  a lack  of  awareness  about  these  grievance  chan- 
nels and  poor  expectation  by  the  people  of  seeing  any  resolutions  come  through, 
as  a result  of  which  these  channels  do  not  capture  implementation  insights  well. 
For  example,  only  17  calls  per  day  are  made  to  the  government  call  centre  in  the 
state  of  Telangana  regarding  (CSOs)  Governments  have  also  set  up  large  MIS  sys- 
tems to  track  minute  details  about  the  implementation  of  various  schemes  to 
bring  more  transparency,  but  these  MIS  systems  are  not  directly  accessible  by  the 
target  beneficiaries  due  to  low  literacy  and  their  inability  to  use  the  Internet,  thus 
impacting  ways  in  which  the  MIS  data  can  be  put  to  better  use.  There  are  provi- 
sions for  conducting  social  audits  but  it  is  a resource  intensive  exercise  in  terms 
of  time,  money  and  human  resources.  Moreover,  independent  social  audit  bodies 
are  missing  in  Telangana  state  of  India. 

Given  the  deep  penetration  of  mobile  phones  in  Telangana,  and  the  ability  of 
even  poorly  literate  people  to  use  basic  calling  functions  and  voice  communica- 
tion, we  argue  that  mobile  phones  can  help  citizens  engage  directly  with  govern- 
ment to  provide  feedback  targeted  at  improving  the  performance  of  welfare 
schemes.  In  this  paper,  we  describe  our  experience  of  using  automated  voice 
calls  via  IVR  systems  to  capture  community  perceptions  and  feedback,  and 
involve  the  communities  in  verifying  the  reported  status  of  activities  done  under 
these  government  schemes.  Our  objective  is  to  verify  broadly  if  technology  can 
provide  a scalable  method  for  citizen  engagement  in  a rural  low-literacy  scenario 
at  this  stage  we  do  not  specially  look  in  detail  at  how  the  IVR  system  should  be 
designed  for  better  usability  or  best  ways  for  integrating  it  with  the  MIS . 

2.  DEVELOPMENTAL  AND  WELFARE  INITIATIVES 

Farmers  in  the  Telangana  state  have  faced  severe  financial  crisis  due  to  frequent 
crop  failures,  lack  of  adequate  support  price,  steep  hike  in  input  costs  and  the 
indifferent  attitude  of  the  erstwhile  United  Andhra  Pradesh  state  government. 
Unable  to  clear  the  heavy  compounding  private  debts  as  well  as  the  loans  raised 
from  the  financial  institutions,  coupled  with  the  day  to  day  expenditure  that  the 
family  demanded  and  driven  to  a despair  situation,  some  farmers,  unfortunately, 
have  resorted  to  suicides.  The  Telangana  Government,  which  took  the  reins,  felt 
that,  it  is  its  responsibility  to  share  burden  of  the  farmers.  Towards  implementing 
its  election  promise,  the  Telangana  government  has  decided  to  waive  up  to 
Rs.  1 .00  Lakh  crop  loans  taken  by  the  farmers.  Agricultural  loans  on  gold  mort- 
gage have  also  been  included,  much  to  the  relief  of  the  farming  community  in  the 
state.  The  initiative  taken  by  the  Telangana  State  government  would  benefit 
about  40  Lakh  families.  Though  the  burden  on  the  government  is  to  the  tune  of 
Rs.18  thousand  Crores,  keeping  in  view  of  the  welfare  of  the  farming  families, 
Honorable  Chief  Minister  K Chandrasekhar  Rao,  who  is  aware  of  the  difficulties 


faced  by  farmers,  took  the  decision  to  waive  the  crop  loans,  there  by  heralding 
good  times  to  the  entire  farming  community  in  the  state. 

A)  Provision  of  Input  Subsidy 

Benevolent  that  K Chandrasekhar  Rao  known  for,  he  did  not  stop  at  waiving  the 
farming  loans,  his  government  had  also  decided  to  pay  the  pending  dues  to  the 
farmers.  The  government  has  released  input  subsidy  to  the  farmers  in  all  the 
Telangana  districts.  The  earlier  governments  did  not  take  note  of  the  requests 
made  by  the  farmers  to  compensate  the  losses  incurred  mainly  due  to  heavy  rains 
resulting  in  damage  to  the  crops.  The  earlier  governments  did  not  respond  at  all 
despite  Dhamas,  Rasta  Rokos  organized  by  the  farmers  to  give  vent  to  their  frus- 
trations. The  present  Telangana  government  has  decided  to  compensate  the 
losses  to  the  farmers.  An  amount  of  Rs.404.58  Crores  has  been  released  as  input 
subsidy  to  the  farmers. 

B)  Allotment  of  3 acres  of  land  to  Dalits  F amilies 

Ever  since  Independence,  there  has  been  lot  of  propaganda  that  the  successive 
governments  have  taken  up  number  of  welfare  programmes  to  benefit  the  Dalits 
and  other  depressed  classes.  In  effect  there  has  not  been  much  of  a change  for  the 
improvement  of  their  lives  except  for  pep  talk  by  the  powers  that  be.  Even  now  in 
every  village  the  worst  downtrodden  people  belong  to  Dalits  community.  There 
are  still  many  Dalits  families  who  cannot  afford  a square  meal  a day.  Majority  of 
Dalits  and  their  family  members  work  as  agriculture  laborers  in  the  un-organized 
sector.  The  Telangana  government  desires  to  change  their  status  into  a full- 
fledged  farming  community.  This  is  the  precise  reason  why  the  Chief  Minister 
decided  to  provide  3 acres  of  land  to  every  Dalit  family.  The  Government  feels 
that  it  is  not  just  enough  to  give  mere  land  but  also  to  provide  facilities  like  bore 
well,  motor,  power  supply  as  well  as  initial  investment  for  a year.  It  is  not  just 
enough  to  talk  about  inequalities  and  untouchables.  This  has  to  be  exhibited  in 
true  letter  and  spirit  by  elevating  the  agricultural  laborers  as  farmers.  The  govern- 
ment feels  that  their  economic  status  needs  to  be  enhanced  and  also  their  respect 
in  the  society  is  to  be  kept  up.  The  Honorable  Chief  Minister  has  taken  up  this 
responsibility  personally  on  his  shoulders.  He  kept  the  welfare  portfolio  with  him 
and  changed  the  name  of  'Social  Welfare  Department'  as  'Scheduled  Caste 
Development  Department'.  On  the  Independence  Day  the  distribution  com- 
menced with  CM  handing  over  pattas  to  Dalit  women  at  the  Golconda  Fort. 

3.  BACKGROUND  ON  CSOs 

CSOs  enacted  to  reduce  rural  poverty  and  stem  rural-urban  migration.  The  Act 
guarantees  a minimum  of  1 00  days  of  work  to  every  rural  household  willing  to  do 
manual  labour.  Job  Cards  are  issued  to  households.  The  intention  is  for  people  to 
demand  work  to  be  done  in  their  village,  which  is  then  discussed  in  a gram  sabha, 
and  approved  by  higher  government  authorities.  Typical  works  include  the  con- 
struction of  roads,  toilets,  water  ponds,  or  planting  of  trees.  No  heavy  machinery 
is  allowed,  and  the  work  cannot  be  outsourced  to  contractors.  An  unemployment 
allowance  is  paid  if  a household  is  not  provided  the  minimum  number  of  guaran- 
teed work-days.  There  is  also  provision  for  basic  work-site  facilities  like  drink- 
ing water  and  toilets.  Wages  must  be  paid  within  1 4 days  from  the  date  of  comple- 
tion of  the  assigned  work  otherwise  a delay  compensation  allowance  is  paid. 
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The  scheme  suffers  from  several  implementation  problems.  Many  people  do  not 
have  job  cards  and  neither  do  they  know  that  it  is  their  right  to  have  a job  card  and 
demand  work.  There  are  also  problems  of  fudged  records,  ghost  assets  and  sub- 
standard quality  of  work.  There  are  cases  where  works  do  not  exist  on  the  ground 
but  the  money  has  been  spent.  Other  common  problems  include  the  hiring  of 
heavy  machines  to  accomplish  the  work,  innate  accounts,  favoritism  in  work 
allotment  and  wrong,  delayed  or  nonpayment  of  wages. 

Our  work  outlines  ways  to  bridge  these  gaps  using  community  driven  and  crowd- 
sourced  reporting  tools  that  can  be  used  by  the  target  beneficiaries  even  though 
many  of  them  are  poorly  literate.  Through  structured  and  unstructured  surveys  to 
collect  community  feedback,  the  implementing  agencies  can  understand  the  com- 
munity's perception  about  the  schemes,  their  level  of  awareness,  and  problems 
they  face  in  accessing  the  schemes.  Through  crowd  sourced  verification,  the 
authorities  can  potentially  even  identify  locations  where  corrupt  practices  are 
rife.  This  will  help  them  to  modify,  plan  and  deploy  the  scheme  in  a better  way 
and  take  corrective  actions  where  necessary. 

4.  IVR  DESIGN 

Our  goal  is  to  design  and  evaluate  an  interactive  voice  response  system  that  per- 
forms an  automatic  “job  interview”  with  low-income  job  seekers.  Potential 
employers,  thereby  making  it  easier  for  them  to  connect  with  qualified  appli- 
cants, could  browse  the  results  of  the  interview.  In  this  work,  we  limit  our  focus  to 
the  domain  of  drivers:  individuals  seeking  full-time  employment  driving  a vehi- 
cle for  a family  or  a company.  To  develop  a questionnaire  that  is  both  meaningful 
for  employers  as  well  as  comprehensible  to  drivers,  we  employed  an  iterative 
design  process.  We  obtained  an  initial  list  of  questions  by  surveying  online 
advertisements  by  those  looking  for  drivers  on  Babajob.com  (a  mobile  and 
Internet  jobs  portal  in  India).  We  refined  this  list  via  in-person  conversations  with 
several  people  who  have  hired  drivers  in  the  past,  including  the  transportation 
manager  at  a large  corporate  office.  Our  resulting  prototype  questionnaire 
encompassed  numeric  questions  (e.g.,  what  is  your  expected  salary?),  yes/no 
questions  (e.g.,  are  you  married?),  multiplechoice  questions  (e.g.,  what  is  your 
level  of  education?),  as  well  as  free-response  questions  (e.g.,  how  would  you  nav- 
igate to  an  unknown  location?).  Numeric,  yes/no,  and  multiplechoice  responses 
were  encoded  as  DTMF  key  presses,  while  free  responses  were  recorded  in  the 
user's  own  voice.  To  evaluate  our  prototype,  we  performed  three  Wizardof-Oz  tri- 
als with  drivers  at  a corporate  office,  as  well  as  six  automated  IVR  trials  that  were 
administered  to  registered  drivers  in  an  online  jobs  portal.  Based  on  the  experi- 
ences of  these  initial  users,  we  improved  the  questionnaire  in  several  ways.  The 
most  important  learnings  were  as  follows: 

• Anticipate  that  users  may  be  in  noisy  or  distracting  environments.  Some 
of  our  participants  had  difficulty  hearing  the  prompts  because  they  received 
our  call  in  a noisy  environment,  or  were  unable  to  give  it  their  full  attention. 
We  responded  by  offering  such  users  the  option  of  rescheduling  the  call  to  a 
time  that  would  be  quieter  or  less  busy.  Also,  we  amended  the  IVR  to  advise 
users  that  each  prompt  will  be  repeated  if  they  do  not  answer;  thus,  in  the 
event  of  distractions  or  noise,  they  can  wait  for  the  repeat  instead  of  trying  to 
answer  a question  they  do  not  understand. 

• Illustrate  multi-digit  entry  using  multiple  examples.  Users  had  difficulty 
understanding  multidigit  entry  (e.g.,  salary  field),  even  when  an  example 
was  provided  by  the  IVR.  Sometimes  users  did  not  respond  at  all,  and  some- 
times they  entered  the  exact  digits  that  were  used  in  the  example  (even 
though  it  did  not  correspond  to  their  salary).  To  alleviate  this  problem,  we  pro- 
vided multiple  examples  of  multi-digit  entry  in  the  instructions. 

• Explain  that  'O'  is  a valid  answer  to  certain  numeric  questions.  For  exam- 
ple, on  the  question  “how  many  times  have  you  received  a traffic  ticket?”, 
participants  did  not  know  what  they  should  enter  if  they  have  never  received 
a ticket.  We  clarified  the  prompt  to  instruct  them  to  press  zero  if  they  have 
never  received  a ticket.  (In  retrospect,  an  even  better  solution  could  have 
been  to  first  ask  whether  or  not  they  had  received  any  tickets,  and  if  so,  to  ask 
for  the  exact  number  of  tickets.)  As  another  example,  for  a question  asking 
how  many  passengers  are  usually  in  the  driver's  car,  a participant  did  not 
know  how  to  answer  because  he  drove  commercial  goods  instead  of  people. 
We  eventually  decided  to  remove  this  question. 


Table  1:  Data  collected  in  our  IVR  questionnaire  for  those  seeking 
employment  as  drivers.  All  questions  were  worded  in  the  local  language 
(Telugu). 


I.  Personal  information 

1.  Age 

multi-digit* 

2.  Marital  status 

yes/no 

3.  Education 

multiple  choice* 

II.  Professional  information 

4.  Own  commercial  permit 

yes/no 

5.  Years  as  a driver 

multi-digit* 

6.  Years  with  license 

multi-digit* 

7.  Number  of  hours  willing  to  work  (per  day) 

multi-digit* 

8.  Open  to  working  night-shifts 

multiple  choice 

9.  Open  to  working  part-time  or  short-term 

yes/no 

10.  Latest  salary  (Rs  / month) 

multi-digit* 

1 1 . Expected  salary  (Rs  / month) 

multi-digit* 

12.  Own  vehicle  for  commute  to  work 

yes/no 

13.  Carry  mobile  phone 

yes/no 

14.  Knowledge  test:  Is  Lenin  Sarani  one-way 

yes/no 

15.  Knowledge  test:  Is  MG  Road  one-way 

yes/no 

16.  Knowledge  test:  Landmark  near  Esplanade 

multiple  choice 

17.  Comfortable  working  outside  Kolkata 

yes/no 

18.  Comfortable  wearing  uniform  to  work 

yes/no 

19.  Comfortable  driving  foreigner 

yes/no 

20.  Willing  to  do  odd  jobs  in  addition  to  driving 

yes/no 

2 1 . Number  of  traffic  tickets  received 

multi-digit* 

22.  Smoking  habits 

multiple  choice* 

23.  Drinking  habits 

multiple  choice* 

III.  Free  response 

24.  Languages  understood,  spoken,  written,  read 

free  response* 

25.  How  to  find  an  unknown  place 

free  response* 

* indicates  questions  with  replay  and  confirmation  of  response 


• Define  all  terms,  leaving  no  room  for  interpretation.  Because  users  do  not 
have  the  opportunity  to  ask  for  clarification  during  the  IVR  interview,  all 
potentially  ambiguous  terms  need  to  be  explained  fully.  For  example,  a par- 
ticipant did  not  know  if  “night  shift”implied  working  continuously  over- 
night, or  working  just  a few  hours  at  night.  As  another  example,  for  a ques- 
tion asking  for  years  of  education,  a participant  did  not  understand  that  their 
3 -year  diploma  after  course  12  counted  as  “more  than  12  years”  of  education. 
We  clarified  all  such  language  to  be  more  explicit. 

Incorporating  the  feedback  from  initial  trials,  we  arrived  at  the  final  design 
for  our  IVR  system,  which  is  illustrated  in  Table  1 . Our  final  questionnaire 
had  25  questions:  7 questions  required  a multi-digit  response,  5 were  multi- 
ple-choice questions,  1 1 were  yes/no  questions,  and  2 required  open,  spoken 
responses.  Three  questions  (#14-#  16)  tested  drivers'  knowledge  of  local 
roads,  while  the  others  gathered  basic  personal  and  professional  information. 
Multiple-choice  questions  had  either  three  or  four  choices.  To  improve  data 
accuracy,  the  system  required  confirmation  for  all  of  the  multi-digit 
responses,  as  well  as  two  multiple-choice  responses  and  both  of  the  oral 
responses.  The  questionnaire  was  developed  in  Bengali,  recorded  by  a native 
Bengali  speaker  and  administered  only  to  native  speakers  of  Telugu.  The  IVR 
system  was  implemented  using  a variant  of  IVR  Junction  [33],  based  on 
Voxeo  Prophecy,  Classic  ASP,  and  IIS  Server.  We  used  a GSM  modem  (Ma- 
trix ATA  2 1 1 G)  with  a mobile  SIM  card  for  the  telephony  interface. 


• Enable  users  to  skip  sensitive  questions.  For  sensitive  questions  (“do  you 
smoke?”  and  “do  you  drink?”),  some  users  were  disconnecting  the  call 
instead  of  answering.  Thus,  we  added  an  option  to  skip  three  questions. 
Potential  employers  could  obtain  this  information  separately  during  a fol- 
low-up interview. 

• Prevent  users  from  going  too  fast.  Because  some  questions  included  an 
explicit  confirmation  (“press  1 to  confirm  your  answer,  or  press  2 to 
change”),  one  participant  wrongly  assumed  that  all  questions  were  followed 
by  confirmation.  Thus,  he  pressed  “1”  after  every  response,  causing  him  to 
answer  some  questions  without  even  listening  to  them.  We  responded  by  pref- 
acing each  new  question  with  a prompt  that  says,  “next  question”,  and  has 
barge-in  disabled. 


Structured  Survey  of  calls  data 

Having  collected  a mass  of  issues  through  Mobile  Vaani,  the  authorities  might 
like  to  probe  them  in  depth  through  some  structured  and  quantitative  questions 
on  these  issues.  For  this  purpose  we  used  an  IVR  based  survey  tool  called  survey 
[12]  developed  by  Gram  Vaani.  Survey  presents  a set  of  voice  questions  over  a 
phone  call,  which  can  be  answered  using  either  DTMF  key  presses,  or  by  ver- 
bally recording  the  answers. 

We  developed  the  questionnaire  for  survey  with  the  help  of  an  expert  from  the  par- 
ent organization  of  our  partner  radio  station  with  inputs  from  the  feedback  com- 
ing into  Mobile  Vaani.  For  example,  we  formulated  a question  on  machine  usage 
in  CSOs  after  we  received  the  feedback  on  Mobile  Vaani  that  in  certain  villages 
machines  were  being  used  instead  of  the  workers  being  given  employment.  We 
asked  a mix  of  yes-no  (binary)  questions,  which  could  be  answered  using  the 
DTMF  buttons  1 and  2,  and  verbal  answer  questions,  which  could  be  answered 
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by  recording  the  response  during  the  call.  The  complete  set  of  questions  is  given 
in  Table  1 . These  questions  are  not  exhaustive  but  are  intended  to  showcase  the 
utility  of  the  platform  to  collect  structured  feedback,  which  can  be  used  to  probe 
qualitative  insights  or  community  perceptions  in  greater  depth. 


Call  Summary  Report 

Call  Status 

Count 

Minutes 

Average 

Transfer 

Average 

1 (Web  Info) 

14 

1585 

1.13 

000 

0 00 

2 (Print  Info)  - complete 

119 

20298 

1.71 

000 

0 00 

2 (Print  Info)  - no  recording 

25 

3045 

1 22 

000 

0 00 

2 (Print  lnfo7)  ■ complete 

1 

1 42 

1 42 

0 00 

0 00 

3 (Public)  complete 

422 

1167  62 

2.77 

000 

o oo] 

3 (Public)  incomplete 

1 

205 

205 

000 

0 00 

3 (Public)  incomplete 

325 

574  52 

1.77 

000 

0 00 

3 (Public)  • no  recording 

242 

26305 

1.09 

000 

0 00 

4 (Custodian)  ■ complete 

282 

670  1 3 

238 

000 

0 00 

4 (Custodian)  - incomplete 

126 

191  70 

1 52 

0 00 

ooo] 

4 (Custodian)  no  recording 

79 

8237 

1 04 

0 00 

0 00 

5 (Reporter  News)  - complete 

31 

74  53 

2 40 

0 00 

0 00 

5 (Reporter  News)  - incomplete 

23 

43  78 

1 90 

0 00 

0 00 

5 (Reporter  News)  no  recording 

45 

51.67 

1.15 

000 

0 00 

Main  Menu  Prompt  - hangup 

277 

349  18 

1 26 

000 

0 00 

Off  Hours  Message 

41 

2370 

058 

000 

0 00 

Rotary  Cal  - complete 

1 

278 

278 

0 00 

0 00 

Rotary  Cal  - complete 

273 

471.53 

1.73 

0 00 

000| 

Rotary  Cal  no  recording 

238 

127  92 

0 54 

0 00 

0 00 

Touch  Tone  Prompt  - hangup 

102 

2337 

023 

0 00 

0 00 

Totals 

2667 

4370.60 

1.64 

0.00 

0.00 
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Fig:  Call  data  report 


5.  CONCLUSION 

We  presented  a proof  of  concept  of  using  IVR  based  tools  to  citizen  engagement 
around  welfare  schemes  in  Telangana.  These  tools  can  help  authorities  get 
insights  about  gaps  in  awareness  of  beneficiaries  on  details  of  the  scheme,  collect 
suggestions  on  modifications  for  the  scheme,  and  understand  implementation 
problems.  The  tools  can  also  be  used  to  verify  the  implementation  and  quality  of 
the  schemes  by  cross  checking  official  data  directly  with  the  beneficiaries.  While 
these  concepts  were  developed  and  evaluated  in  the  context  of  CSOs,  they  can  be 
easily  extended  to  collect  feedback  and  verify  details  about  other  welfare 
schemes  such  as  IVR  system  was  implemented  using  a variant  of  IVR  Junction 
and  the  public  distribution  system  as  well.  We  see  a strong  case  to  advocate  the 
integration  of  such  systems  as  part  of  the  government  schemes  themselves,  to 
enable  the  beneficiaries  to  directly  access  their  own  records  and  provide  feed- 
back. 
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